Attachable matter detection apparatus

ABSTRACT

A decision part decides a representative edge direction using a predetermined angle range as a unit for each pixel area of a plurality of pixel areas of a photographic image photographed by a camera based on an edge angle of each pixel contained in the pixel area. An extractor extracts two of the pixel areas that are adjacent to each other as a pair area among the pixel areas of the photographic image, when the two pixel areas that are adjacent to each other have opposite representative edge directions. A determination part determines whether or not there is an attachable matter on a lens of the camera based on at least one of (1) a number of the pair areas extracted by the extractor and (2) a total sum of edge intensities of the pixel areas contained in the pair areas.

BACKGROUND OF THE INVENTION Field of the Invention

The invention relates to an attachable matter detection apparatus and anattachable matter detection method.

Description of the Background Art

Conventionally, there is known an attachable matter detection apparatusthat detects an attachable matter attached to a lens of a camera basedon a photographic image photographed by the camera mounted on a vehicle,or the like. The attachable matter detection apparatus detects anattachable matter, for example, based on a difference betweentime-series photographic images.

SUMMARY OF THE INVENTION

According to one aspect of the invention, an attachable matter detectionapparatus includes a controller configured to function as a decisionpart, an extractor and a determination part. The decision part decides arepresentative edge direction using a predetermined angle range as aunit for each pixel area of a plurality of pixel areas of a photographicimage photographed by a camera. The representative edge direction isdetermined for each of the pixel areas based on an edge angle of eachpixel contained in the pixel area. The extractor extracts two of thepixel areas that are adjacent to each other as a pair area among thepixel areas of the photographic image, when the two pixel areas that areadjacent to each other have opposite representative edge directions. Thedetermination part determines whether or not there is an attachablematter on a lens of the camera based on at least one of (1) a number ofthe pair areas extracted by the extractor and (2) a total sum of edgeintensities of the pixel areas contained in the pair areas.

As a result, it is possible to detect an attachable matter early andwith high accuracy.

Therefore, an object of the invention is to provide an attachable matterdetection apparatus and an attachable matter detection method capable ofdetecting an attachable matter early and with high accuracy.

These and other objects, features, aspects and advantages of theinvention will become more apparent from the following detaileddescription of the invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an overview of an attachable matter detection methodaccording to an embodiment;

FIG. 2 is a block diagram illustrating a configuration of an attachablematter detection apparatus according to the embodiment;

FIG. 3 illustrates a process of a decision part;

FIG. 4 illustrates a process of the decision part;

FIG. 5 illustrates a process of a determination part;

FIG. 6 illustrates a process of the determination part;

FIG. 7 illustrates a process of the determination part;

FIG. 8 is a flowchart illustrating a processing procedure of a detectionprocess of an attachable matter executed by the attachable matterdetection apparatus according to the embodiment;

FIG. 9 illustrates a process of a decision part according to amodification; and

FIG. 10 illustrates a process of an extractor according to themodification.

DESCRIPTION OF THE EMBODIMENTS

An attachable matter detection apparatus and an attachable matterdetection method according to an embodiment of the present applicationwill now be described in detail with reference to the accompanyingdrawings. The present disclosure is not limited to the embodimentdescribed in the following.

First, an overview of the attachable matter detection method accordingto the embodiment will be described with reference to FIG. 1. FIG. 1illustrates the overview of the attachable matter detection methodaccording to the embodiment. FIG. 1 shows a photographic image Iphotographed, for example, in a state in which snow is attached on anentire surface of a lens of an in-vehicle camera. The attachable matterdetection method according to the embodiment is executed by anattachable matter detection apparatus 1 (refer to FIG. 2), and detectsan attachable matter attached to the lens of the in-vehicle camera basedon the photographic image I photographed by the in-vehicle camera.

The attachable matter is not limited to snow, and may be, for example,light-colored dirt, or the like. In other words, although, in thephotographic image I, the attachable matter prevents an object frombeing reflected, some light may transmit through the attachable matterand a small luminance change may be caused by a light transmissionvariation.

Here, in a conventional attachable matter detection apparatus, there isa technology that detects an attachable matter based on a differencebetween times-series photographic images. However, in the conventionaltechnology, for example, if the entire lens is covered with snow, or thelike, the difference between the images is hardly caused so that thereis a possibility that an attachable matter cannot be detected.Conventionally, in a photographic image covered with snow, for example,when a vehicle travels in a tunnel, a part of the photographic image Itemporarily has high luminance due to a light source in the tunnel. Inthis case, since an intensity of an edge increases in an area of highluminance due to a light source, erroneous detection of a state in whichthere is no attachable matter may be performed. Furthermore,conventionally, since times-series photographic images are required, ittakes times to detect an attachable matter.

Therefore, in the attachable matter detection method according to theembodiment, an attachable matter is detected using an edge angle that isdetected from a single photographic image I. Specifically, asillustrated in FIG. 1, the attachable matter detection apparatus 1according to the embodiment first decides a representative edgedirection using a predetermined angle range as a unit and an edgeintensity for each pixel area 100 based on the edge angle (a vectordirection within each pixel PX in FIG. 1) of each pixel PX contained inthe pixel area 100 of the photographic image I (a step S1).

For example, the pixel area 100 is an area in which the pixels PX arearranged in a 4×4 matrix (16 pixels PX in total). In FIG. 1, forconvenient description purposes, only a single pixel area 100 is shown,but actually, in the entire photographic image I (or a part of thephotographic image I in which an attachable matter is detected), aplurality of the pixel areas 100 is set in a two-dimensional (verticaland horizontal) array. A setting range of the pixel areas 100 may be theentire photographic image I or a center area obtained by excluding avehicle body area from the photographic image I.

The representative edge direction is an edge direction representing therespective edge angles of 4×4 pixels PX. In FIG. 1, the representativeedge direction is divided into four directions by each angle range of90°, and one of the four representative edge directions is decided. Theedge intensity is an edge intensity representing the respective edgeintensities of 4×4 pixels. A decision process of the representative edgedirection and edge intensity will be described below with reference toFIG. 3 and FIG. 4.

Subsequently, in the attachable matter detection method according to theembodiment, among the pixel areas 100, when two pixel areas 100 adjacentto each other have opposite representative edge directions, the twopixel areas 100 adjacent to each other are extracted as a pair area 200(a step S2).

In the example of FIG. 1, a pair area 200 a in which two pixel areas 100are adjacent to each other in a vertical direction and a pair area 200 bin which two pixel areas 100 are adjacent to each other in a horizontaldirection are shown. An extraction process of the pair area 200 a andthe pair area 200 b will be described below. If the pair area 200 a isnot particularly distinguished from the pair area 200 b, the pair area200 obtained by deleting the last digit is described.

Subsequently, in the attachable matter detection method according to theembodiment, it is determined whether or not there is an attachablematter on the lens based on at least one of a number of the extractedpair areas 200 and a total sum of the edge intensities of the pixelareas 100 contained in the pair area 200 (a step S3).

The number of the pair areas 200 refers to a total value obtained byadding the number of the pair areas 200 a each in which two pixel areas100 are adjacent to each other in the vertical direction to the numberof the pair areas 200 b each in which two pixel areas 100 are adjacentto each other in the horizontal direction. The total sum of the edgeintensities of the pixel areas 100 in the pair areas 200 refers to atotal value obtained by adding up the edge intensities of all of thepixel areas 100 contained in the pair areas 200. Calculation methods ofthe number of the pair areas 200 and the total sum of the edgeintensities will be described below with reference to FIG. 5 and FIG. 6.

For example, when there is no attachable matter, a relatively largenumber of the pair areas 200 are extracted due to a white line on theroad, a building outline, etc. Furthermore, since the edge intensitiesof the pixel areas 100 are high, the total sum of the edge intensitiesof the pixel areas 100 in the pair areas 200 relatively increases. Onthe other hand, when the entire lens is covered with an attachablematter, since the luminance of the photographic image I is totallyuniform and the edge intensities of the pixel areas 100 decrease, boththe number of the pair areas 200 to be extracted and the total sum ofthe edge intensities of the pixel areas 100 in the pair areas 200relatively decrease.

Therefore, by focusing on this point, as illustrated in FIG. 1, in theattachable matter detection method according to the embodiment, when thenumber of the pair areas 200 is equal to or more than a predeterminedthreshold value TH10, and the total sum of the edge intensities is equalto or more than a predetermined threshold value TH20, it is determinedthat there is no attachable matter on the lens (a non-attached state).On the other hand, in the attachable matter detection method accordingto the embodiment, when the number of the pair areas 200 is less thanthe predetermined threshold value TH10, and the total sum of the edgeintensities is less than the predetermined threshold value TH20, it isdetermined that there is an attachable matter on the lens.

In the attachable matter detection method according to the embodiment,it may be further determined whether an attachable matter is attachedpartially on the lens or attached on the entire lens. However, this willbe described in detail below with reference to FIG. 5.

As described above, in the attachable matter detection method accordingto the embodiment, by detecting an attachable matter based on the numberof the pair areas 200 or the total sum of the edge intensities, evenwhen the difference between the images is not caused, it is possible todetect an attachable matter with high accuracy. For example, even when apart of the photographic image I temporarily has high luminance due to alight source in the tunnel, and the like, a change in the number of thepair areas 200 and the total sum of the edge intensities is totallyminute. As a result, it is possible to reduce erroneous detection of astate in which there is no attachable matter despite the fact there isan attachable matter. Furthermore, in the attachable matter detectionmethod according to the embodiment, it is possible to detect anattachable matter using a single photographic image I. Therefore,according to the attachable matter detection method according to theembodiment, it is possible to detect an attachable matter early and withhigh accuracy.

FIG. 1 illustrates a case where a single type of representative edgedirection is decided for each pixel area 100, but, for example, byfurther deciding representative edge directions having different angleranges, two or more types of representative edge directions may bedecided for each pixel area 100. A case where two types ofrepresentative edge directions are decided will be described below withreference to FIG. 9.

FIG. 1 illustrates a case where it is determined whether or not there isan attachable matter based on the number of the pair areas 200(predetermined threshold value TH10) or the total sum of the edgeintensities (predetermined threshold value TH20). For example, on atwo-dimensional map represented by the number of the pair areas 200 andthe total sum of the edge intensities, map information in which areasfor determining whether “there is an attachable matter” or “there is noattachable matter” are set may be created in advance and it may bedetermined whether or not there is an attachable matter based such mapinformation.

Next, a configuration of the attachable matter detection apparatus 1according to the embodiment will be described with reference to FIG. 2.FIG. 2 is a block diagram illustrating the configuration of theattachable matter detection apparatus 1 according to the embodiment. Asillustrated in FIG. 2, the attachable matter detection apparatus 1according to the embodiment is connected to a camera 10 and variousdevices 50. FIG. 2 illustrates a case where the attachable matterdetection apparatus 1 is configured separately from the camera 10 andthe various devices 50. However, the invention is not limited thereto.The attachable matter detection apparatus 1 may be configured integrallywith at least one of the camera 10 and the various devices 50.

The camera 10 is, for example, an in-vehicle camera including a lens,such as a fish-eye lens, and an image sensor, such as a charge coupleddevice (CCD) or a complementary metal oxide semiconductor (CMOS). Thecamera 10 is, for example, provided at each position capable ofphotographing front, rear and side images of a vehicle, and outputs thephotographed photographic image I to the attachable matter detectionapparatus 1.

The various devices 50 acquire a detection result of the attachablematter detection apparatus 1 and perform various control of the vehicle.The various devices 50 include, for example, a display that notifies auser that an attachable matter is attached on the lens of the camera 10and instructs the user to remove the attachable matter from the lens, aremover that removes the attachable matter by spraying a fluid, air,etc. to the lens, and a vehicle control device that controls autonomousdriving, and the like.

As illustrated in FIG. 2, the attachable matter detection apparatus 1according to the embodiment includes a controller 2 and a memory 3. Thecontroller 2 includes an image acquisition part 21, a decision part 22,an extractor 23, and a determination part 24. The memory 3 storesthreshold value information 31.

Here, the attachable matter detection apparatus 1 includes a computerhaving, for example, a central processing unit (CPU), a read only memory(ROM), a random access memory (RAM), a data flash, an input/output port,and the like, and various circuits.

The CPU of the computer serves as the image acquisition part 21, thedecision part 22, the extractor 23, and the determination part 24 of thecontroller 2, for example, by reading and executing a program stored inthe ROM.

At least any one or all of the image acquisition part 21 the decisionpart 22, the extractor 23, and the determination part 24 of thecontroller 2 may be configured of hardware such as an applicationspecific integrated circuit (ASIC) or a field programmable gate array(FPGA).

The memory 3 corresponds to, for example, the RAM or data flash. The RAMor data flash may store the threshold value information 31, orinformation on various programs. The attachable matter detectionapparatus 1 may acquire the programs or information described above viacomputers connected via a wired or wireless network or a portablerecoding medium.

The controller 2 decides the representative edge direction for eachpixel area 100 of the photographic image I and determines whether or notthere is an attachable matter based on the number of the pair areas 200each in which two pixel areas 100 adjacent to each other have oppositerepresentative edge directions, and the total sum of the edgeintensities of the pixel areas 100 contained in the pair areas 200.

The image acquisition part 21 acquires the photographic image Iphotographed by the camera 10. The image acquisition part 21 performs agrayscale conversion of expressing each pixel of the acquiredphotographic image I in gray scales from white to black depending on theluminance, performs a smoothing process on each pixel, and outputs theprocessed pixel to the decision part 22. For example, an averagingfilter or an arbitrary smoothing filter, such as a Gaussian filter, maybe used for the smoothing process. The grayscale conversion and thesmoothing process may be omitted.

The decision part 22 decides the representative edge direction for eachpixel area 100 of the photographic image I acquired from the imageacquisition part 21. Here, the decision process of the representativeedge direction by the decision part 22 will be specifically describedwith reference to FIG. 3 and FIG. 4.

FIG. 3 and FIG. 4 illustrate a process of the decision part 22. Asillustrated in FIG. 3, the decision part 22 first performs an edgedetection process to detect an intensity of an edge ex in the X-axisdirection (horizontal direction of the photographic image I) and anintensity of an edge ey in the Y-axis direction (vertical direction ofthe photographic image I). For example, a Sobel filter or an arbitraryedge detection filter such as a Prewitt filter may be used for the edgedetection process.

Subsequently, the decision part 22 calculates a vector V includinginformation of the edge angle and the edge intensity of each pixel PXusing a trigonometric function based on the detected edge intensity ofthe edge ex in the X-axis direction and the detected edge intensity ofthe edge ey in the Y-axis direction. Specifically, an angle θ betweenthe vector V and the X-axis on the positive direction side will bereferred to as an edge angle, and the length L of the vector will bereferred to as an edge intensity of each pixel.

Subsequently, the decision part 22 decides the representative edgedirection in the pixel area 100 based on the calculated vector V of eachpixel PX. Specifically, as illustrated in FIG. 4, each pixel PX (havingan angle range of −180° to 180°) is divided into four parts (each havingan angle range of 90°), and the four parts (groups) are indicated bycodes “(0)” to “(3)”, respectively.

Specifically, when the edge angle in the vector V is within an anglerange of −45° or more and less than 45°, the decision part 22 classifiesthe edge angle into the group “(0)”. When the edge angle in the vector Vis within an angle range of 45° or more and less than 135°, the decisionpart 22 classifies the edge angle into the group “(1)”. When the edgeangle in the vector V is within an angle range of 135° or more and lessthan 180° or within an angle range of −180° or more and less than −135°,the decision part 22 classifies the edge angle into the group “(2)”.When the edge angle in the vector V is within an angle range of −135° ormore and less than −45°, the decision part 22 classifies the edge angleinto the group “(3)”.

As illustrated in a lower stage of FIG. 4, the decision part 22 createsa histogram using each of the groups “(0)” to “(3)” as each of gradesfor each pixel area 100. When a frequency of a grade whose frequency isthe highest in the created histogram is a predetermined threshold valueTHa or more, the decision part 22 decides the group corresponding to thegrade (group “(1)” in FIG. 4) as the representative edge direction inthe pixel area 100. That is, the decision process of the representativeedge direction by the decision part 22 can be regarded as a process ofencoding each pixel area 100.

The frequency of the histogram is calculated by adding up the edgeintensities of the pixels PX which are classified into a same anglerange among the pixels PX within the pixel area 100. Specifically, thefrequency of the histogram that belongs to the group (grade) “(0)” willbe considered. For example, there are three pixels PX which areclassified into the group (grade) “(0)”, and the edge intensities of thethree pixels PX are 10, 20, and 30, respectively. In this case, thefrequency of the histogram that belongs to the group (grade) “0” iscalculated by the following equation: 10+20+30=60. The frequency of thehistogram that belongs to the group (grade) “(1)”, “(2)”, or “(3)” iscalculated in a similar manner.

Based on the calculated histogram, the decision part 22 decides the edgeintensity. Specifically, when the frequency of the grade whose frequencyis the highest in the histogram is the predetermined threshold value THaor more, the frequency corresponding to the grade is regarded as theedge intensity. That is, a decision process of a representative edgeintensity by the decision part 22 can be regarded as a process ofextracting a characteristic related to the edge intensity within thepixel area 100 corresponding to the representative edge direction.

On the other hand, when the frequency of the grade whose frequency isthe highest is less than the predetermined threshold value THa, thedecision part 22 regards the representative edge direction in the pixelarea 100 as “invalid”, in other words, “no representative edgedirection”. As a result, when there is a large variation in the edgeangles of each pixel PX, it is possible to prevent an erroneousdetermination of a specific representative edge direction.

The decision process of the decision part 22 illustrated in FIG. 3 andFIG. 4 is merely one example, and the process may be arbitrary if theprocess can decide the representative edge direction. For example, amean value of the edge angles of each pixel PX in the pixel area 100 iscalculated to decide the representative edge direction in the anglerange corresponding to the mean value.

FIG. 4 illustrates a case where a single pixel area 100 consists of 16(4×4) pixels PX, but a number of the pixels PX in the pixel area 100 maybe arbitrarily set and may be different in vertical and horizontaldirections, such as 3×5 pixels.

Referring back to FIG. 2, the extractor 23 will be described. Among thepixel areas 100 of the photographic image I, when two pixel areas 100adjacent to each other have opposite representative edge directions, theextractor 23 extracts the two pixel areas 100 which are adjacent to eachother as the pair area 200.

Specifically, the extractor 23 scans a plurality of the pixel areas 100of the photographic image I which are arranged in a two-dimensional(horizontal and vertical) array in horizontal and vertical directions,and searches for the pair area 200. That is, the extractor 23 extractstwo pixel areas 100 adjacent to each other in the horizontal or verticaldirection that is the scanning direction. The extractor 23 outputsinformation of the extracted pair area 200 to the determination part 24.

The determination part 24 determines whether or not there is anattachable matter on the lens based on at least one of the number of thepair areas 200 extracted by the extractor 23 and the total sum of theedge intensities of the pixel areas 100 contained in the pair areas 200.For example, the determination part 24 determines whether or not thereis an attachable matter based on the threshold value information 31stored in the memory 3.

Here, a determination process of the determination part 24 will bespecifically described with reference to FIG. 5 to FIG. 7. Each of FIG.5 to FIG. 7 illustrates the determination process of the determinationpart 24. FIG. 5 and FIG. 6 illustrate the calculation methods of thenumber of the pair areas 200 and the total sum of the edge intensities.FIG. 7 illustrates the determination process using the calculated numberof the pair areas 200 and the calculated total sum of the edgeintensities.

FIG. 5 illustrates a case where there is no overlapping pixel area 100between two pair areas 200. FIG. 6 illustrates a case where there is asingle overlapping pixel area 100 between two pair areas 200. Asillustrated in FIG. 5, when there is no overlapping pixel area 100between the extracted two pair areas 200, the determination part 24determines that the number of the pair areas 200 is two and the totalsum of the edge intensities is a total value obtained by adding up theedge intensities of four pixel areas 100.

On the other hand, as illustrated in FIG. 6, when there is a singleoverlapping pixel area 100 between two pair areas 200, the determinationpart 24 determines that the number of the pair areas 200 is two and thetotal sum of the edge intensities is a total value obtained by adding upthe edge intensities of the pixels PX contained in three pixel areas100. That is, the determination part 24 does not regard an overlappingpixel area 100 as an independent pixel area 100 in each of the two pairareas 200, and calculates the total sum of the edge intensities assumingthat the overlapping pixel area 100 is a single pixel area 100.

The determination part 24 may regard an overlapping pixel area 100 as anindependent pixel area 100 in each of the two pair areas 200. That is,the determination part 24 may calculate the total sum of the edgeintensities assuming that the overlapping pixel area 100 is two pixelareas 100. When there is a single overlapping pixel area 100 between twopair areas 200, the total sum of the edge intensities is obtained byadding up the edge intensities of four pixel areas 100.

As illustrated in FIG. 7, the determination part 24 determines whetheror not there is an attachable matter based on whether or not thecalculated number of the pair areas 200 and the total sum of the edgeintensities of the pixel areas 100 contained in the pair areas 200satisfy the predetermined threshold values. In FIG. 7, the predeterminedthreshold values TH10, TH11, TH12, TH20 and TH21 included in thethreshold value information 31 are shown. The threshold valueinformation 31 is made in advance, for example, by experiments.

Specifically, when the number of the pair areas 200 is equal to or morethan the predetermined threshold value TH10 and the total sum of theedge intensities of the pixel areas 100 contained in the pair areas 200is equal to or more than the predetermined threshold value TH20, thedetermination part 24 determines that a non-attached state exists inwhich there is no attachable matter on the lens (“there is no attachablematter” shown in FIG. 7). On the other hand, when the number of the pairareas 200 is less than the predetermined threshold value TH10 and thetotal sum of the edge intensities of the pixel areas 100 contained inthe pair areas 200 is less than the predetermined threshold value TH20,the determination part 24 determines that there is an attachable matteron the lens.

That is, it is determined whether or not there is an attachable matterbased on whether or not a lot of characteristics of a contour of anobject such as a white line, a building, or the like are seen in theentire photographic image I. As a result, for example, even when a partof the photographic image I temporarily has high luminance due to alight source in the tunnel, and the like, a change in the number of thepair areas 200 obtained from the entire photographic image I and thetotal sum of the edge intensities is minute. Thus, it is possible toreduce erroneous detection of a state in which there is no attachablematter despite the fact there is an attachable matter.

As illustrated in FIG. 7, by setting the predetermined threshold valuesTH11, TH12 and TH21 different from the predetermined threshold valuesTH10, TH20, it is also possible to determine whether an attachablematter is attached partially on the lens or attached on the entiresurface of the lens.

Specifically, when the number of the pair areas 200 is less than thepredetermined threshold value TH10 and equal to or more than thepredetermined threshold value TH11, and the total sum of the edgeintensities of the pixel areas 100 contained in the pair areas 200 isless than the predetermined threshold value TH20 and equal to or morethan the predetermined threshold value TH21, the determination part 24determines that a partially attached state exists in which theattachable matter is attached partially on the lens.

On the other hand, when the number of the pair areas 200 is less thanthe predetermined threshold value TH11 and equal to or more than thepredetermined threshold value TH12, and the total sum of the edgeintensities of the pixel areas 100 contained in the pair area 200 isless than the predetermined threshold value TH21, the determination part24 determines that an entirely attached state exists in which theattachable matter is attached on an entirety of the lens.

For example, in the photographic image I photographed in a state inwhich an attachable matter is attached on the entire lens, since anobject does not have any contours and the luminance of the photographicimage I is totally uniform, both the number of the pair areas 200 andthe total sum of the edge intensities of the pixels PX contained in thepair areas 200 extremely decrease. On the other hand, in thephotographic image I photographed in a state in which an attachablematter is attached partially on the lens, since an object has someslight contours and the luminance of the photographic image I is nottotally uniform, both the number of the pair areas 200 and the total sumof the edge intensities of the pixel areas 100 contained in the pairareas 200 increase compared to the photographic image I photographed inthe state in which an attachable matter is attached on the entire lens.

That is, the determination part 24 determines whether or not there is anattachable matter using image characteristics that appear depending onthe attached state of an attachable matter on the lens. As a result, thedetermination part 24 can determine with high accuracy whether anattachable matter is attached partially on the lens or attached on theentire lens.

When the number of the pair areas 200 is less than the predeterminedthreshold value TH12, the determination part 24 determines that thephotographic image I is totally in a black screen state. That is, whenthe number of the pair areas 200 is less than the predeterminedthreshold value TH12, the determination part 24 does not determinewhether or not there is an attachable matter.

This is because the entire photographic image I temporarily becomes ablack screen state by a user removing an attachable matter on the lensand it cannot be determined whether or not there is an attachable matterin the black screen. As described above, when the number of the pairareas 200 is less than the predetermined threshold value TH12, it ispossible to prevent erroneous detection of an attachable matter by notperforming the determination process of an attachable matter.

FIG. 7 illustrates an attachable matter determination in which both thenumber of the pair areas 200 and the total sum of the edge intensitiesof the pixel areas 100 contained in the pair areas 200 are used.However, the attachable matter determination may be performed using atleast one of the number of the pair areas 200 and the total sum of theedge intensities of the pixel areas 100 contained in the pair areas 200.

Next, a processing procedure executed by the attachable matter detectionapparatus 1 according to the embodiment will be described with referenceto FIG. 8. FIG. 8 is a flowchart illustrating the processing procedureof a detection process of an attachable matter executed by theattachable matter detection apparatus 1 according to the embodiment.

As illustrated in FIG. 8, the image acquisition part 21 acquires thephotographic image I, and performs a grayscale conversion and asmoothing process on the photographic image I (a step S101).

Subsequently, the decision part 22 decides the representative edgedirection using a predetermined angle range as a unit for each pixelarea 100 of a plurality of pixel areas 100 of the photographic image Iphotographed by the camera 10, and the representative edge direction isdetermined for each of the pixel areas based on an edge angle of eachpixel PX contained in the pixel area 100 (a step S102).

Subsequently, among the pixel areas 100 of the photographic image I,when two of the pixel areas 100 that are adjacent to each other haveopposite representative edge directions, the extractor 23 extracts thetwo pixel areas 100 that are adjacent to each other as the pair area 200(a step S103).

Subsequently, the determination part 24 calculates the number of thepair areas 200 extracted by the extractor 23 and the total sum of theedge intensities of the pixel areas 100 contained in the pair areas 200(a step S104). The determination part 24 may use at least one of thenumber of the pair areas 200 extracted by the extractor 23 and the totalsum of the edge intensities of the pixel areas 100 contained in the pairareas 200 for the processes in steps S105 to S111.

Subsequently, the determination part 24 determines whether or not thenumber of the pair areas 200 is equal to or more than the predeterminedthreshold value TH10, and the total sum of the edge intensities of thepixel areas 100 contained in the pair areas 200 is equal to or more thanthe predetermined threshold value TH20 (the step S105).

When the number of the pair areas 200 is equal to or more than thepredetermined threshold value TH10 and the total sum of the edgeintensities of the pixel areas 100 contained in the pair areas 200 isequal to or more than the predetermined threshold value TH20 (Yes in thestep S105), the determination part 24 determines that a non-attachedstate exists in which there is no attachable matter on the lens (thestep S106), and ends the process.

On the other hand, when the number of the pair areas 200 is less thanthe predetermined threshold value TH10 and the total sum of the edgeintensities of the pixel areas 100 contained in the pair areas 200 isless than the predetermined threshold value TH20 (No in the step S105),the determination part 24 determines whether or not the number of thepair areas 200 is equal to or more than the predetermined thresholdvalue TH11 and the total sum of the edge intensities of the pixel areas100 contained in the pair areas 200 is equal to or more than thepredetermined threshold value TH21 (the step S107).

When the number of the pair areas 200 is equal to or more than thepredetermined threshold value TH11 and the total sum of the edgeintensities of the pixel areas 100 contained in the pair areas 200 isequal to or more than the predetermined threshold value TH21 (Yes in thestep S107), the determination part 24 determines that a partiallyattached state exists in which the attachable matter is attachedpartially on the lens (the step S108), and ends the process.

On the other hand, when the number of the pair areas 200 is less thanthe predetermined threshold value TH11 and the total sum of the edgeintensities of the pixel areas 100 contained in the pair areas 200 isless than the predetermined threshold value TH21 (No in the step S107),the determination part 24 determines whether or not the number of thepair areas 200 is equal to or more than the predetermined thresholdvalue TH12 (the step 5109).

When the number of the pair areas 200 is equal to or more than thepredetermined threshold value TH12 (Yes in the step S109), thedetermination part 24 determines that an entirely attached state existsin which the attachable matter is attached on an entirety of the lens(the step S110), and ends the process.

On the other hand, when the number of the pair areas 200 is less thanthe predetermined threshold value TH12 (No in the step S109), thedetermination part 24 determines that the photographic image I is in ablack screen state (the step S111), and ends the process.

As described above, the attachable matter detection apparatus 1 includesthe decision part 22, the extractor 23, and the determination part 24.The decision part 22 decides the representative edge direction using apredetermined angle range as a unit for each pixel area 100 based on theedge angle of each pixel PX contained in the pixel area 100 of thephotographic image I. Among the pixel areas 100 of the photographicimage I, when two pixel areas 100 adjacent to each other have oppositerepresentative edge directions, the extractor 23 extracts the two pixelareas 100 adjacent to each other as the pair area 200. The determinationpart 24 determines whether or not there is an attachable matter on thelens of the camera 10 based on at least one of the number of the pairareas 200 extracted by the extractor 23 and the total sum of the edgeintensities of the pixel areas 100 contained in the pair areas 200. As aresult, it is possible to detect an attachable matter early and withhigh accuracy.

In the embodiment described above, a single type of representative edgedirection is decided for each pixel area 100, but two or more types ofrepresentative edge directions may be decided for each pixel area 100.This will be described with reference to FIG. 9 and FIG. 10.

FIG. 9 illustrates a process of a decision part 22 according to amodification. FIG. 10 illustrates a process of an extractor according tothe modification. In FIG. 9 and FIG. 10, a process of deciding two typesof representative edge directions having different angle ranges will bedescribed.

As illustrated in FIG. 9, the decision part 22 decides a firstrepresentative edge direction using a first angle range as a unit foreach of the pixel areas 100 and a second representative edge directionusing a second angle range different from the first angle range as aunit for each of the pixel areas 100.

Specifically, the decision part 22 divides each pixel PX (having anangle range of −180° to 180°) into four parts (each having the firstangle range of 90°), and the four parts (groups) are indicated by codes“(0)” to “(3)”, respectively. Furthermore, the decision part 22 divideseach pixel PX (having an angle range of −180° to 180°) into four parts(each having the second angle range of 90° different from the firstangle range), and the four parts (groups) are indicated by codes “(4)”to “(7)”, respectively.

More specifically, when an edge angle in a vector V is within an anglerange of −45° or more and less than 45°, the decision part 22 classifiesthe edge angle into the group “(0)”. When the edge angle in the vector Vis within an angle range of 45° or more and less than 135°, the decisionpart 22 classifies the edge angle into the group “(1)”. When the edgeangle in the vector V is within an angle range of 135° or more and lessthan 180° or within an angle range of −180° or more and less than −135°,the decision part 22 classifies the edge angle into the group “(2)”.When the edge angle in the vector V is within an angle range of —135° ormore and less than −45°, the decision part 22 classifies the edge angleinto the group “(3)”

Furthermore, when the edge angle in the vector V is within an anglerange of 0° or more and less than 90°, the decision part 22 classifiesthe edge angle into the group “(4)”. When the edge angle in the vector Vis within an angle range of 90° or more and less than 180°, the decisionpart 22 classifies the edge angle into the group “(5)”. When the edgeangle in the vector V is within an angle range of 180° or more and lessthan −90°, the decision part 22 classifies the edge angle into the group“(6)”. When the edge angle in the vector V is within an angle range of−90° or more and less than 0°, the decision part 22 classifies the edgeangle into the group “(7)”.

As illustrated in a lower stage of FIG. 9, the decision part 22 creates,for each pixel area 100, a histogram using each of the groups “(0)” to“(3)” as each of grades and a histogram using each of the groups “(4)”to “(7)” as each of grades. When a frequency of a grade whose frequencyis the highest in the created histogram is a predetermined thresholdvalue THa or more, the decision part 22 decides the group “(0)”corresponding to the grade as the first representative edge direction,and the decision part 22 decides the group “(4)” corresponding to thegrade as the second representative edge direction.

As illustrated in FIG. 10, there are two pixel areas 100 a and 100 bwhich are adjacent to each other. As described above, the pixel area 100a has the first representative edge direction and the secondrepresentative edge direction decided by the decision part 22.Similarly, the pixel area 100 b has the first representative edgedirection and the second representative edge direction decided by thedecision part 22. When at least one of the following conditions is met:(i) the respective first representative edge directions of the two pixelareas 100 a and 100 b are opposite to each other; and (ii) therespective second representative edge directions of the two pixel areas100 a and 100 b are opposite each other, the extractor 23 extracts thetwo pixel areas 100 a and 100 b that are adjacent to each other as thepair area 200.

That is, for each pixel area 100, by deciding the first and secondrepresentative edge directions, it is possible to extract the pair area200 which is not extracted only by a single type of representative edgedirection.

For example, one pixel has an edge angle of 140° and the other pixel hasan edge angle of −40°. Although the first representative edge directionsof the two pixels are not opposite to each other, the secondrepresentative edge directions of the two pixels are opposite to eachother. Thus, it is possible to detect a change in the edge angle of thepixel area 100 with higher accuracy.

In the embodiment and modification described above, in each pixel PXhaving an angle range of −180° to 180°, the representative edgedirection is divided into four directions by each angle range of 90°.However, the angle range is not limited to 90°. For example, therepresentative edge direction may be divided into six directions by eachangle range of 60°.

Widths of the respective angle ranges for the first representative edgedirection and the second representative edge direction may be differentfrom each other. For example, the first representative edge directionmay be divided into four directions by each angle range of 90°, and thesecond representative edge direction may be divided into six directionsby each angle range of 60°.

A boundary of two adjacent angle ranges for the first representativeedge direction is deviated by an angle of 45° from a boundary of twoadjacent ranges for the second representative edge direction. However,the deviated angle may exceed 45°, or may be less than 45°. It ispossible to arbitrarily set the boundaries of two adjacent angle rangesfor the first representative edge direction and the secondrepresentative edge direction.

In the embodiment described above, although the photographic image Iphotographed by the camera to be mounted on the vehicle is used, thephotographic image I photographed by a security camera or a camera on astreet light may be used. That is, the photographic image I photographedby a camera with a lens to which an attachable matter can be attachedmay be used.

In the embodiment described above, although the determination part 24determines whether or not there is an attachable matter based on thepredetermined threshold values TH10 and TH20, for example, detection ofan attachable matter may be performed by totally comprehending adetection result of the attachable matter by other algorithms and adetermination result by the determination part 24. Specifically, otheralgorithms determine that there is an attachable matter, and thedetermination part 24 according to the embodiment may determine thatthere is no attachable matter after the attachable matter is, forexample, removed.

It is possible for a person skilled in the art to easily come up withmore effects and modifications. Thus, a broader modification of thisinvention is not limited to specific description and typical embodimentsdescribed and expressed above. Therefore, various modifications arepossible without departing from the general spirit and scope of theinvention defined by claims attached and equivalents thereof.

While the invention has been shown and described in detail, theforegoing description is in all aspects illustrative and notrestrictive. It is therefore understood that numerous othermodifications and variations can be devised without departing from thescope of the invention.

What is claimed is:
 1. An attachable matter detection apparatuscomprising a controller configured to function as: a decision part thatdecides a representative edge direction using a predetermined anglerange as a unit for each pixel area of a plurality of pixel areas of aphotographic image photographed by a camera, the representative edgedirection being determined for each of the pixel areas based on an edgeangle of each pixel contained in the pixel area; an extractor thatextracts two of the pixel areas that are adjacent to each other as apair area among the pixel areas of the photographic image, when the twopixel areas that are adjacent to each other have opposite representativeedge directions; and a determination part that determines whether or notthere is an attachable matter on a lens of the camera based on at leastone of (1) a number of the pair areas extracted by the extractor and (2)a total sum of edge intensities of the pixel areas contained in the pairareas.
 2. The attachable matter detection apparatus according to claim1, wherein the determination part determines that a non-attached stateexists in which there is no attachable matter on the lens when at leastone of the number of the pair areas and the total sum of the edgeintensities of the pixel areas contained in the pair areas is equal toor more than a first predetermined threshold value.
 3. The attachablematter detection apparatus according to claim 2, wherein thedetermination part determines that a partially attached state exists inwhich the attachable matter is attached partially on the lens when atleast one of the number of the pair areas and the total sum of the edgeintensities of the pixel areas contained in the pair areas is less thanthe first predetermined threshold value and equal to or more than asecond predetermined threshold value.
 4. The attachable matter detectionapparatus according to claim 3, wherein the determination partdetermines that an entirely attached state exists in which theattachable matter is attached on an entirety of the lens when at leastone of the number of the pair areas and the total sum of the edgeintensities of the pixel areas contained in the pair areas is less thanthe second predetermined threshold value and equal to or more than athird predetermined threshold value.
 5. The attachable matter detectionapparatus according to claim 1, wherein the decision part decides (A) afirst representative edge direction using a first angle range as a unitfor each of the pixel areas and (B) a second representative edgedirection using a second angle range different from the first anglerange as a unit for each of the pixel areas, and the extractor extractsthe two pixel areas that are adjacent to each other as the pair areawhen at least one of the following conditions is met: (i) the respectivefirst representative edge directions of the two pixel areas are oppositeto each other; and (ii) the respective second representative edgedirections of the two pixel areas are opposite to each other.
 6. Anattachable matter detection method comprising the steps of: (a)deciding, by a controller, a representative edge direction using apredetermined angle range as a unit for each pixel area of a pluralityof pixel areas of a photographic image photographed by a camera, therepresentative edge direction being determined for each of the pixelareas based on an edge angle of each pixel contained in the pixel area;(b) extracting, by the controller, two of the pixel areas that areadjacent to each other as a pair area among the pixel areas of thephotographic image, when the two pixel areas that are adjacent to eachother have opposite representative edge directions; and (c) determining,by the controller, whether or not there is an attachable matter on alens of the camera based on at least one of (1) a number of the pairareas extracted by the step (b) and (2) a total sum of edge intensitiesof the pixel areas contained in the pair areas.